Geospatial information creating system and geospatial information creating method

ABSTRACT

To be precisely extracted a house footprint. There is provided a geospatial information creating system for extracting a footprint of a house from an aerial photograph, comprising a processor for executing a program, a memory for storing data required for executing the program, and a storage unit for storing the aerial photograph. The processor detects edges of an image based on a characteristic quantity of neighboring pixels in the aerial photograph stored in the storage unit; extracts an orientation of the house by analyzing directions of the detected edges; and generates a polygon of an outline of the house by using linear lines of the extracted orientation of the house.

CLAIM OF PRIORITY

The present application claims priority from Japanese patent applicationJP 2009-224204 filed on Sep. 29, 2009, the content of which is herebyincorporated by reference into this application.

BACKGROUND OF THE INVENTION

This invention relates to a geospatial information creating system, andmore particularly, to a technology of extracting a house footprint fromaerial photograph and satellite imagery.

Geospatial information of houses (specifically house footprint) is veryimportant for cadastral management. It is normally highly expensive toacquire such information through either conventional ground survey ormanual digitization of aerial photograph. Extraction of house footprintsfrom remote sensing image data (which includes, in addition to aerialphotograph, satellite imagery and both of them are hereaftercollectively referred to as “aerial photograph”) is a very promisingsolution for updating geospatial information system (GIS) data. However,this task has remained a challenge in past decades due to its highcomplexity.

Though intensive efforts to extract house footprint have been conductedin past decades, there still is no automatic tool proposed for reliablyextracting house footprint, as described in Gulch, E. and Muller, H. Newapplications of semi-automatic building acquisition. In AutomaticExtraction of Man-Made Objects From Aerial and Space Images (III),Edited by Emmanuel P. Baltsavias, Armin Gruen, Luc Van Gool. ISBN:90-5809-252-6. by A.A. BALKEMA PUBLISHERS, pp. 103-114. 2001. Theconventional methods generally fall into two categories according totheir primitive features used. One is based on edge-linking to linkedges of houses along house boundaries. Another one is based on regiongrowth algorithm to generate house regions by merging similarneighboring pixels.

However, both methods are very much sensitive to noise and incapable ofdealing with complex cases such as weak edges and occlusion, thereforethese methods are only applicable to very few practical cases.Currently, although semi-automatic systems are most efficient, they canhardly generate a house footprint map with correct geometric shape.

Mayunga, S. D., Zhang, Y. and Coleman, D. J. Semi-automatic buildingextraction utilizing Quickbird imagery. In Stilla, U. Rottensteiner, F.and Hinz, S. (Eds) CMRT05. IAPRS, Vol. XXXVI, Part 3/W24—Vienna,Austria, Aug. 29-30, 2005 describes a method based on dynamic contourmodel (for example, snake model), which deforms contour throughminimizing an energy function, and generates an initial contour using“radial casting” algorithm. This method allows semi-automatic extractionof buildings from satellite imagery (for example, image taken byQuickbird). However, house footprints extracted according to “radialcasting” algorithm are coarse and irregular, and hence the method cannot solve the problem.

US 2004/0263514 A1 describes a semi-automatic method to extract houseboundaries by linking the edges near the initial points given by user,but the method described in US 2004/0263514 A1 is insufficient todetermine whether the edges really belong to target houses.

JP 2003-346144 A describes a method to use air-borne laser scanning datafor segmenting house regions and then combining image data to extracthouse footprints. However, the method described in JP 2003-346144 A willvery much increase the data cost by using laser scanning and alsointroduces uncertainty by combing data from different sources.

SUMMARY OF THE INVENTION

In extraction of house footprints from aerial photographs, detection ofa house and extraction of the house are difficult.

The goal of house detection is to locate where a house is and to segmentthe house from other objects. In aerial photograph, houses are difficultto separate from its surrounding objects particularly in urban areas,where parking lots, grass land, bare fields, water bodies, roadnetworks, and other man-made objects etc. exhibit very similar featuresto houses, such as shape, texture, spectral signature, etc. So far ithas not been seen what kind of features or their combinations actuallyprovide effective “general rules” for detecting and extracting houses.

The term “house extraction” is abused in the art. Many conventionalmethods and systems claim to extract houses, but actually justidentify/find locations or coarse windows of houses. In this invention,house detection is explicitly defined to derive house footprint outlinesand represent them geometrically.

It is very challenging to extract house boundaries with an acceptablegeometric accuracy because houses have a wide variety of rooftop shapesand sub-structures. Moreover, assumable disturbance also comes fromshadows, shadings, noises, and occlusions from other objects such astrees and other structures. The conventional “Box model” is a popularmethod which treats a house as a rectangular box, but it is a simplemethod for depicting house shapes and results in very coarseapproximation of houses.

A geospatial information creating system for extracting a footprint of ahouse from an aerial photograph, comprising a processor for executing aprogram, a memory for storing data required for executing the program,and a storage unit for storing the aerial photograph. The processordetects edges of an image based on a characteristic quantity ofneighboring pixels in the aerial photograph stored in the storage unit;extracts an orientation of the house by analyzing directions of thedetected edges; and generates a polygon of an outline of the house byusing linear lines of the extracted orientation of the house.

According to the representative embodiment of this invention, the housefootprints can be precisely extracted.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention can be appreciated by the description whichfollows in conjunction with the following figures, wherein:

FIG. 1 is a block diagram illustrating structure of a geospatialinformation creating system according to an embodiment of thisinvention;

FIG. 2 is a flowchart illustrating overall house footprint extractionprocessing according to the embodiment of this invention;

FIG. 3A is an image for describing removal of landmarks based on aspectrum according to the embodiment of this invention;

FIG. 3B is an image for describing removal of vegetation based on thespectrum according to the embodiment of this invention;

FIG. 3C is an image for describing removal of roads according to theembodiment of this invention;

FIG. 3D is an image for describing removal of water bodies based on thespectrum according to the embodiment of this invention;

FIG. 4 is a table for describing processing carried out based onselection by a user according to the embodiment of this invention;

FIG. 5 is a flowchart illustrating, in detail, house extractionprocessing according to the embodiment of this invention;

FIG. 6 is a flowchart illustrating, in detail, angle histogramgeneration processing according to the embodiment of this invention;

FIG. 7A is an image for describing the angle histogram generationprocessing according to the embodiment of this invention;

FIG. 7B is a histogram for describing the angle histogram generationprocessing according to the embodiment of this invention;

FIG. 7C is a histogram for describing the angle histogram generationprocessing according to the embodiment of this invention;

FIG. 8 is a flowchart illustrating, in detail, house orientationextraction processing according to the embodiment of this invention;

FIG. 9 is a flowchart illustrating, in detail, Straight Line Segment(SLS) classification processing according to the embodiment of thisinvention;

FIG. 10 is a flowchart illustrating, in detail, grid generationprocessing according to the embodiment of this invention;

FIG. 11 is a flowchart illustrating, in detail, grid line extractionprocessing according to the embodiment of this invention;

FIGS. 12A and 12B are images for describing a deviation r from a gridline according to the embodiment of this invention;

FIGS. 13A and 13B are images for describing classification of SLSsaccording to the embodiment of this invention;

FIG. 13C is an image for describing generation of grids according to theembodiment of this invention;

FIG. 14 is a flowchart illustrating processing for calculating asimilarity index of grid cells according to the embodiment of thisinvention;

FIG. 15 illustrates a house rooftop processed by the geospatialinformation creating system according to the embodiment of thisinvention;

FIG. 16 illustrates an image of the house rooftop processed by thegeospatial information creating system according to the embodiment ofthis invention;

FIG. 17 illustrates linear edges of the house rooftop extracted by thegeospatial information creating system according to the embodiment ofthis invention;

FIG. 18 illustrates SLSs classified into a class perpendicular to ahouse orientation by the geospatial information creating systemaccording to the embodiment of this invention;

FIG. 19 illustrates grid lines extracted by the geospatial informationcreating system according to the embodiment of this invention;

FIG. 20 illustrates SLSs classified into a class parallel with the houseorientation by the geospatial information creating system according tothe embodiment of this invention;

FIG. 21 illustrates grid lines extracted by the geospatial informationcreating system according to the embodiment of this invention;

FIG. 22 illustrates a house area partitioned into grids with the gridlines in the geospatial information creating system according to theembodiment of this invention;

FIG. 23 illustrates a rooftop area of the house generated by merginggrid cells in the geospatial information creating system according tothe embodiment of this invention;

FIG. 24 illustrates a polygon in a shape of a house footprint generatedby the geospatial information creating system according to theembodiment of this invention;

FIGS. 25A to 25H are images for describing a step of the house footprintextraction according to the embodiment of this invention; and

FIG. 26 illustrates house footprints extracted according to theembodiment of this invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT Configuration of System

FIG. 1 is a block diagram illustrating structure of a geospatialinformation creating system according to an embodiment of thisinvention.

The geospatial information creating system according to this embodimentis a computer including an input unit 110, a displaying unit 120, animage storage unit 130, a processing unit 140, a map output unit 150,and a storage unit 160. The input unit 110, the displaying unit 120, theimage storage unit 130, the map output unit 150, and the storage unit160 are coupled to each other via the processing unit 140 (or coupledthrough a bus).

The input unit 110 comprises an image input unit 111, and a positionaldata input unit 112. The image input unit 111 is a device through whichan image of houses is input, and is configured by an optical disk drive,a USB interface, and the like, for example. The positional data inputunit 112 is a user interface through which an operator enters aninstruction, and is configured by a pointing device such as a mouse, akeyboard, and the like.

The display unit 120 comprises an image displaying unit 121 and a mapdisplaying unit 122. The image displaying unit 121 is a display devicefor displaying an image to be processed by the geospatial informationcreating system. The map displaying unit 122 is a display device fordisplaying a map generated by the geospatial information creatingsystem. It should be noted that the image displaying unit 121 and themap displaying unit 122 may be configured by the same display device ordifferent display devices.

The image storage unit 130 is a storage device for storing images to beprocessed by the geospatial information creating system, and isconfigured by a hard disk drive, a nonvolatile memory, and the like, forexample. The processing unit 140 is a processing device (a processor),and, by executing programs, executes processing carried out in thissystem.

The map output unit 150 is a device for outputting maps generated by thegeospatial information creating system, and is configured by a printer,plotter, and the like, for example. The storage unit 160 is a storagedevice for storing programs to be executed by the processing unit 140,map information, and a database of house footprints, and is configuredby a hard disk drive, a nonvolatile memory, and the like, for example.

<Processing>

FIG. 2 is a flowchart illustrating overall house footprint extractionprocessing according to the embodiment of this invention.

First, an image of houses (for example, an aerial photograph) receivedby the image input unit 111 is analyzed according to spectrum andgeometric shapes for removing vegetation, water bodies, and roads fromthe image. Specifically, the vegetation such as trees, forests, andgrasses emits light in a specific spectrum. Thus, areas having thisspecific spectrum can be distinguished from the photograph image. Thespectrum of the vegetation can be analyzed by the well-known normalizeddifference vegetation index (NDVI) method using light in a plurality ofchannels such as visible (red) and infrared light. Moreover, the waterbody such as pools, ponds, lakes, canals, rivers, etc. has a near-zeroNDVI value, and thus these areas are removed from the photograph image.On the other hand, road, bare-land and building/house etc. have anambiguous spectrum. However, roads have long linear continuousgeometrical features, and thus areas having these geometrical shapes areremoved from the photogram image (S11).

FIGS. 3A to 3D describe the removal of landmarks based on the spectrum.

FIG. 3B illustrates vegetation segmented by the NDVI method, FIG. 3Cillustrates roads segmented according to the geometrical shapes (longlinear line), and FIG. 3D illustrates water bodies segmented by the NDVImethod. In Step S11, by using, as a mask image, an image (FIG. 3A)obtained by combining the vegetation (FIG. 3B), the roads (FIG. 3C), andthe water bodies (FIG. 3D), the vegetation, the roads, and the waterbodies can be removed from the aerial photograph.

Thereafter, specification of an initial position (P(X, Y)) in a housewhose footprint is to be extracted is received on the image through thepositional data input unit 112 (S21). Next, an image in a predeterminedrange containing the specified initial position is carved out (S22).This carved-out area is a Region of Interest (ROI). The ROI confines aworking space, and thus accelerates processing.

Thereafter, in the carved-out ROI, a house area is derived (S23). InStep S23, the derived house area may not exactly align with the houseboundary, but preferably covers the target house and is close to thecontour of the house. Details of the house area derivation processingare described later referring to FIG. 5.

Thereafter, image edges are detected in the derived house area (S31).For this edge detection, a well-known method may be used, and, forexample, an existence of an edge is determined if a change incharacteristic quantity (for example, brightness and color information)between neighboring pixels is larger than a predetermined threshold.

Thereafter, the extracted edges are fitted to Straight Line Segments(SLSs) (S32). Thereafter, a histogram of angles of the SLSs extracted inStep S32 is generated (S33). Details of the histogram generationprocessing are described later referring to FIG. 6.

Thereafter, a house orientation is extracted based on the anglehistogram (S34). Walls and neighboring walls of a man-made housenormally form right or near-right angles. Thus, the extracted SLSs existin a pair of orthogonal directions in a Cartesian coordinate system.Thus, the direction which primary SLSs align with is defined as theorientation of house. Details of the angle histogram generationprocessing are described later referring to FIG. 8.

Thereafter, based on the deviation of the SLSs extracted in Step S32,the SLSs are clustered into two groups of the parallel and perpendiculardirections (S35). Details of the SLS clustering processing are describedlater referring to FIG. 9.

Thereafter, based on the SLSs clustered into the two groups, grids aregenerated (S36). Details of this grid generation processing aredescribed later referring to FIGS. 10 to 12.

Thereafter, spectra of grid cells divided by the respective grids arecalculated, similarity indices of the grid cells are calculated based onthe calculated spectra, and grid cells which are determined to besimilar are marked (S41). Details of the similarity index calculationprocessing are described later referring to FIG. 14.

Thereafter, the similar grid cells are merged to generate rectilinearpolygon patch corresponding to a house footprint surrounded by linearlines (S42). The polygon patch corresponding to a house footprintgenerated in this way is well matched with the shape of a house rooftop.

Thereafter, the outer boundary of the generated polygon patch isretrieved, and polygon patch surrounded by the retrieved outer boundaryis saved into the database as a house footprint (S51). It should benoted that, in Step S51, a user interface may be provided to give achance to determine whether the retrieved boundary is satisfactory orrequires human's adjustment.

Thereafter, in the image, the extracted house footprint area is masked(S52). This mask prevents the masked area from being selected forfurther processing, thereby preventing the processing of deriving thehouse area from being carried out again.

FIG. 4 is a table for describing the processing carried out as a resultof the selection of the user (S52). In the processing of S52, byreferring to a determination table illustrated in FIG. 4, contents ofthe processing is determined.

When the extraction of a house footprint is successful, the area of theextracted house footprint is masked, and this house area is no longersubject to the selection, and is thus not extracted again.

On the other hand, when the extraction of a house footprint fails, thearea is not masked, and hence can be selected again as an image area towhich the processing of derivation of house area is applied again.

Thereafter, an input for whether or not another house is to be extractedis received (S61). When there is an input for extraction of anotherhouse from the user, the processing returns to Step S21, and thespecification of an initial position is received.

On the other hand, when there is an input for finishing the extractionof a house from the user, by reading out the house footprint saved inthe database in Step S51, and writing the read-out house footprint on amap, a map of house footprint is generated (FIG. 26, for example) (S71),and the house footprint extraction processing is finished.

FIG. 5 is a flowchart illustrating details of the house area derivationprocessing (S23) according to the embodiment of this invention.

First, a similarity index S indicating a correlation among pixels in theROI is set to a high value (S231), and an image segmentation method (forexample, Graph Cut) is applied with the similarity index S to segmentthe image in the ROI into regions (S232). Because the high value of Smeans the tight correlation among neighboring pixels, resulting in smallbut homogeneous segmented object regions.

Whether or not the specified position P(X, Y) in Step S21 is containedin a segmented object region is judged thereafter (S233).

When the point P(X, Y) is not contained in any segmented region, itmeans that an region containing the subject house has not yet beenderived. Therefore, the similarity index S is reduced so that even whenthe relationship between neighboring pixels is weak, the pixels aredetermined to be similar (S234). Thereafter, the processing returns toStep S232, the processing of Steps S232 and S233 is repeated until anarea containing the point P(X, Y) is segmented.

On the other hand, when the point P(X, Y) is contained in a segmentedregion, this region is significantly different from a periphery thereof,and is thus an area containing the point P(X,Y). As a result, the areacontaining the point P(X,Y) is output as a house area (S235). It shouldbe noted that the periphery of the house area output in Step S235 may beextended by a predetermined length or area as a house area.

FIG. 6 is a flowchart illustrating details of the angle histogramgeneration processing (S33) according to the embodiment of thisinvention.

First, the length li and the orientation direction θi of each SLS arecalculated (S331). The orientation direction θi is weighted with the SLSlength li as a weight factor so as to determine a contribution value tothe histogram by the SLS length li.

Next, an equation (1) is used to calculate values of the histogram forangles of all the SLSs (S332).

$\begin{matrix}\left\lbrack {{Equation}\mspace{14mu} 1} \right\rbrack & \; \\{{a(\partial)} = {{\sum\limits_{i = 1}^{n}{f\left( l_{i} \right)}}_{\theta = \partial}}} & (1)\end{matrix}$

After the calculation of the histogram value of the angle is repeatedfor all the SLSs, the angle histogram generation processing (S33) isfinished.

FIGS. 7A to 7C are histograms for describing a process of the anglehistogram generation processing (S33).

Linear edges aligned to SLSs are represented as lines as illustrated inFIG. 7A.

As described before, walls and neighboring walls of a man-made housenormally form right or near-right angles, and the footprint of the houseis thus a rectangle. Moreover, buildings with other footprints areusually combination of rectangles. Through investigating the appearanceof houses in aerial photographs, these right-angles form fairly obviousfeatures.

The SLSs illustrated in FIG. 7A are represented as a histogram weightedby the length of the SLSs while the horizontal axis represents the anglethereof in FIG. 7B. Moreover, similarly, a circular histogram weightedby the length of the SLSs is illustrated in FIG. 7C.

According to FIGS. 7B and 7C, two peaks forming a right angle areapparent, and these two peaks most significantly characterize theorientation of the house. Next, one of the angles presenting the higherpeak corresponds to the orientation of the house.

FIG. 8 is a flowchart illustrating details of the house orientationextraction processing (S34) according to the embodiment of thisinvention.

First, respective parameters for executing the house orientationextraction processing are initialized. Specifically, the orientationangle β is set to 0, the maximum histogram value H is set to 0, and asearching angle θ is set 0 (S341).

Thereafter, whether the searching angle θ is equal to or more than 0°and less than 90° is determined (S342). As a result, processing in aloop from Step S343 to Step S346 is repeated within a range of 0≦θ<90.On the other hand, when θ exceeds the range of 0≦θ<90 (namely θ is equalto or more than 90°), the processing proceeds to Step S347.

In Step S343, an equation (2) is used to calculate a histogram valueh(θ) for each orthogonal angle pair. In this description, the orthogonalangle pair is a pair of angles separated +90 degrees or −90 degrees eachother.

$\begin{matrix}\left\lbrack {{Equation}\mspace{14mu} 2} \right\rbrack & \; \\{{h(\theta)} = {{\sum\limits_{{\Delta\varphi} = {- c}}^{+ c}{a\left( {\theta + {\Delta\varphi}} \right)}} + {\sum\limits_{{\Delta\varphi} = {- c}}^{+ c}{a\left( {\theta + 90 + {\Delta\varphi}} \right)}}}} & (2)\end{matrix}$

In the equation (2), histogram values are summed in a range from anangle θ−c to an angle θ+c. This c represents a neighborhood of the angleθ, and the half of a step s of the searching angle described later inStep S346 is preferably used. Further, in the equation (2), to thehistogram values in the neighborhood of the angle θ, histogram values inthe neighborhood of the angle θ+90° are also added. This is based on thefact that neighboring walls of a house are orthogonal to each other.

After the histogram value h(θ) is calculated, h(θ) and the maximumhistogram value H are compared with each other (S344). As a result, whenh(θ) is larger than H, the angle is closer to the house orientationangle, and hence, in order to update the maximum histogram value H, h(θ)is saved to H, and the angle θ which gives h(θ) is set to theorientation angle β (S345). On the other hand, when h(θ) is not largerthan H, it is not necessary to update the maximum histogram value H, andhence the calculated h(θ) is neglected, and the processing proceeds toStep S346.

In Step S346, a new θ is defined by adding the increment value s to theangle θ, the processing returns to Step S342, and h(θ) is calculatedrepeatedly in the range of 0≦θ<90.

In Step S347, the orientation angle β is set to the house orientationangle.

FIG. 9 is a flowchart illustrating details of the SLS clusteringprocessing (S35) according to the embodiment of this invention.

In order to compare the orientation angle θ of the each SLS and thehouse orientation angle β, a deviation δ=|θ−β| is calculated, and thecalculated deviation δ and a predetermined value α are compared witheach other (S351). The value α is a permissible deviation of theorientation angle of the SLS.

As a result, when δ≦α, the orientation angle θ of the SLS is close tothe orientation angle β of the house, and thus the SLS is classified asan SLS parallel with the house orientation (S352). On the other hand,when |δ−90|≦α, the orientation angle θ of the SLS is approximatelyperpendicular to the orientation angle β of the house, and thus the SLSis classified as an SLS perpendicular to the house orientation (S353).Further, when δ>α and |δ−90|>α, this SLS is irrelevant to contours ofthe house, and is thus deleted (S354).

Next, by repeating the processing in Steps S351 to S354, all the SLSsare classified.

FIG. 10 is a flowchart illustrating details of the grid generationprocessing (S36) according to the embodiment of this invention.

First, from the each clustered SLS group in which the groups areclustered into two classes, grid lines in the two directions areextracted (S361). Details of the grid line extraction processing aredescribed later referring to FIG. 11.

Thereafter, the extracted grid lines are made to intersect to partitionthe house area into grids (S362).

FIG. 11 is a flowchart illustrating the details of the grid lineextraction processing (S361) according to the embodiment of thisinvention, and, to each of the directions of the two orthogonalcoordinate axes, the grid line extraction processing is applied.

First, a blank image of the house area is prepared, and, to this blankimage, SLSs parallel with the orientation of the house is output(S3611). Thereafter, for an i row (or column) of the respective SLSs inthe image area, the number h(i) of pixels is calculated (S3612). Thispixel number h(i) represents the length of the each SLS.

Thereafter, the maximum value max[h(i)] of h(i) is calculated, and isset to h_max. Specifically, h(i) and h_max are compared, and when h(i)is equal to or more than h_max, a significant grid line is present nearthe i row, and hence h(i) is set to h_max (S3613).

Thereafter, h_max and t are compared (S3614). t is a predeterminedthreshold, and is used to determine whether an SLS can be a major gridline.

When h_max is less than t, the grid line extraction processing isfinished, and a predetermined value (for example, value twice as largeas r used in Step S3617) is added to i, and a next SLS is processed. Inthis description, a parameter i is sequential number for specifying lineor row. A parameter r determines a buffer range (predeterminedneighboring range). For example, i specifies nth line if i=n. Further,the buffer range of 10th line is n=7 to 13 if i=10 and r=3.

On the other hand, when h_max is equal to or more than t, the SLS is amajor grid line, and hence, according to an equation (3), a histogramvalue H(i) for h(i) of the i row (or column) in the buffer range iscalculated (S3615).

$\begin{matrix}\left\lbrack {{Equation}\mspace{14mu} 3} \right\rbrack & \; \\{{{H(i)} = {\sum\limits_{{\Delta \; s} = {- r}}^{+ r}{{k\left( {\Delta \; s} \right)}*{h\left( {i + {\Delta \; s}} \right)}}}}{{k\left( {\Delta \; s} \right)} = \left\{ \begin{matrix}{0,} & {{{\Delta \; s}} > r} \\{\left( {1 - \frac{{\Delta \; s}}{r}} \right)^{2},} & {{{\Delta \; s}} \leq r}\end{matrix} \right.}} & (3)\end{matrix}$

The precise position of the grid line is determined at row J where H(J)is equal to max[H(i)], and thus, the maximum value of the grid linehistogram is calculated according to an equation (4). J which gives thecalculated H_max is at the highest in density of SLS, the value of J isoutput as the position of the grid line (S3616).

[Equation 4]

H_max=H(J)=max[H(i)]  (4)

When the grid line is output at the J row, nearby SLSs of the J row havealready contributed to the histogram of the grid line of the J row andtheir influence to other grid lines should be reduced correspondingly.Therefore the values of pixel number h(i) of the SLSs near the grid lineare recalculated according to an equation (5) (S3617). On this occasion,as illustrated in FIGS. 12A and 12B, r represents a deviation from agrid line, and SLSs within the range of ±r from the grid line is mergedto the grid line of J row. This calculation reduces the value h(i)inversely to the distance to the J row.

$\begin{matrix}\left\lbrack {{Equation}\mspace{14mu} 5} \right\rbrack & \; \\{{{h(j)} = {\left( \frac{j - i}{r} \right)^{2}*{h(j)}}}{{i - r} \leq j \leq {i + r}}} & (5)\end{matrix}$

Thereafter, the processing returns to Step S3616, the predeterminedvalue (for example, value twice as large as r used in Step S3617) isadded to and a next SLS is processed. In this description, a parameter iis sequential number for specifying line or row.

FIGS. 13A to 13C are images for describing the process in which SLSs areclassified, and a house area is partitioned into grids.

Of SLSs parallel with and perpendicular to the house orientationclassified from the linear edges (SLSs), illustrated in FIG. 7A, FIG.13A illustrates the parallel SLSs and FIG. 13B illustrates theorthogonal SLSs. Thereafter, as illustrated in FIG. 13C, the grid linesperpendicular to and parallel with the house orientation are determined.

FIG. 14 is a flowchart illustrating the processing of calculatingsimilarity indices of grid cells (S41) according to the embodiment ofthis invention.

First, RGB values of pixels in a grid cell are converted into values inthe HSI color space (S411). The conversion between the color spaces iscarried out because the HSI color space is close to a perceived colorspace, and thus, the perceived color difference is well associated withthe Euclidean distance in the HSI color space.

Thereafter, the mean shift (MS) algorithm is applied to segment pixelsin each grid cell image (S412). The MS algorithm is a very versatileprocedure for image segmentation and it can significantly reduce thenumber of basic image clusters.

Thereafter, of the segmented regions, small regions (namely fragmentaland noise like regions) are removed (S413).

Thereafter, according to an equation (6), a mean vector of the grid cellis calculated as the similarity index of the grid cell (S414).

[Equation 6]

{right arrow over (v)}=({right arrow over (h)},{right arrow over(s)},{right arrow over (i)})  (6)

Next, by repeating the processing in Steps S411 to S414, the meanvectors are calculated for all the grid cells.

Thereafter, based on the Euclidean distance between calculated meanvectors, the similarity index for neighboring grid cells is calculated(S415).

A description is now given of how an aerial photography changesaccording to the processing of this invention.

The embodiment of this invention can efficiently extract housefootprints from aerial photographs and satellite imagery. Moreover, themethod according to this embodiment can extract a house footprintwithout depending on any other auxiliary data.

Moreover, not only identifying the location of a house, the method canextract the geometric shape of a house footprint based on therectilinear house model. This enables the processing to significantlysimplify the complexity of house footprints with little loss ofgeometric accuracy.

As illustrated in FIG. 15, a rooftop of a house normally forms ridges,and thus appears complicated. The image of a house rooftop is evenblurred due to the illustration condition and the limits of imagingsensor. As illustrated in FIG. 16, the image of the house rooftop isdrawn on predetermined grids.

As illustrated in FIG. 17, the edges of the image extracted in Step S31align with the physical boundary of the house. Moreover, noises in theimage are also detected as edges.

As described above, the detected linear edges are thereafter fitted intothe SLSs (S32). Thereafter, a histogram is generated in Step S33. Next,a house orientation is extracted based on the angle histogram of SLSs(S34). The SLSs are clustered into two groups as illustrated in FIGS. 18and 20, which are approximately parallel with and perpendicular to theorientation of the house, respectively.

Thereafter, based on the SLSs perpendicular to the house orientation,grid lines in the orthogonal direction illustrated in FIG. 19 areextracted, and, based on the SLSs parallel with the house orientation,grid lines in the parallel direction illustrated in FIG. 21 areextracted (S361).

Thereafter, as illustrated in FIG. 22, the house area is partitioned bygrids generated by using the extracted grid lines.

In FIG. 23, similar grid cells are merged into a rooftop area of thehouse (S41 and S42), and a polygon patch surrounded by the boundaryextracted from the generated rooftop area (FIG. 24) is saved as a housefootprint in the database (S51).

FIGS. 25A to 25G are images for describing extraction of a housefootprint from a real image.

Image edges are extracted from the house area (ROI) carved out from anaerial image (S31, FIG. 25A). Thereafter, the extracted image edges arefitted to SLSs (S32, FIG. 25B). The extracted SLSs are clustered intoSLSs parallel with the house orientation, and SLSs perpendicular to thehouse orientation (S35, FIGS. 25C and 25D).

Thereafter, based on the SLSs clustered into two groups, grids aregenerated (S36, FIG. 25E). Next, similarity indices of the grid cellsgenerated by the grids are calculated, and grid cells determined to besimilar are marked (S41, FIG. 25F). Next, the grid cells determined tobe similar are merged (S42, FIG. 25G), and a rectilinear polygon patchcorresponding to a house footprint surrounded by outlines of the mergedgrid cells are generated (S51, FIG. 25H).

In this way, by displaying the house footprints extracted in a pluralityof house areas on one map, a house footprint map illustrated in FIG. 26can be generated.

When a house footprint is extracted from an aerial photograph, edges ofa house may be weak and occluded by other house footprints, and thehouse edges may thus be partially or totally invisible. In this case,for the conventional method (edge-linking method), the conditions suchas edge direction, right angle etc. defined to guide link of edgesappear weak and are insufficient to determine the correct edgecandidates. Therefore, usually, a house footprint cannot be extracted ina portion in which edges of a house are invisible. On the other hand,region based methods normally fail to accurately locate the boundary ofa house of interest. Specifically, the pixels of the house area do notextend to the edges of the image, and thus, there is no way to “pull”the house area for the edges of the image having occlusion.

The embodiment of this invention integrates the primitive features (suchas edges and regions) into the rectilinear house model. Consequently,even for a complicated case such as a case of weak edges or occludededges, a house geometric model and primitive features are properlymatched. As a result, the embodiment of this invention can preciselyextract a house footprint compared with the conventional technologies.

While the present invention has been described in detail and pictoriallyin the accompanying drawings, the present invention is not limited tosuch detail but covers various obvious modifications and equivalentarrangements, which fall within the purview of the appended claims.

1. A geospatial information creating system for extracting a footprintof a house from an aerial photograph, comprising a processor forexecuting a program, a memory for storing data required for executingthe program, and a storage unit for storing the aerial photograph,wherein the processor is configured to: detect edges of an image basedon a characteristic quantity of neighboring pixels in the aerialphotograph stored in the storage unit; extract an orientation of thehouse by analyzing directions of the detected edges; and generate apolygon of an outline of the house by using linear lines of theextracted orientation of the house.
 2. The geospatial informationcreating system according to claim 1, wherein the processor is furtherconfigured to: receive specification of a position by an operator forthe house to be extract the footprint; determine a house area of anextent of the extraction of the edges by periodically extractingneighboring pixels more similar than a threshold before extraction ofthe edges; and change the threshold to decrease, and extract againneighboring pixels more similar than the changed threshold in the casewhere the determined house area does not include the specified position.3. The geospatial information creating system according to claim 1,wherein the processor is further configured to: determine straight linesegments to which the detected edges fit; define the orientation of thehouse based on angles of the determined straight line segments; classifythe determined straight line segments into those in the definedorientation of the house and those in a direction orthogonal thereto;generate grids at positions at which the classified straight linesegments exist; and generate the polygon of the outline of the houseaccording to the generated grids.
 4. The geospatial information creatingsystem according to claim 3, wherein the processor is further configuredto: calculate a color spectrum in each grid cell segmented by thegenerated grids, and calculate a similarity index of neighboring gridcells based on the calculated color spectrum; and merge grid cellshaving the calculated similarity index larger than a predeterminedthreshold to generate a rectilinear polygon patch corresponding to thefootprint of the house surrounded by linear lines.
 5. The geospatialinformation creating system according to claim 3, wherein the processoris further configured to: determine a first frequency of a position ofthe determined straight line segments which is weighted by a length ofthe determined straight line segments, and generate a grid lineconstructing the grid at a position having the first frequency largerthan a predetermined threshold; and calculate the first frequency sothat the first frequency of an straight line segment close to thegenerated grid line is added to the first frequency of the grid line. 6.The geospatial information creating system according to claim 3, whereinthe processor is further configured to determine a second frequency ofthe angle of the determined straight line segments which is weighted bya length of the determined line, and define an angle high in the secondfrequency as the orientation of the house.
 7. A geospatial informationcreating method for extracting a footprint of a house from an aerialphotograph, the method being carried out by a computer system comprisinga processor for executing a program, a memory for storing data requiredfor executing the program, and a storage unit for storing the aerialphotograph, the method including the steps of: detecting edges of animage based on a characteristic quantity of neighboring pixels in theaerial photograph stored in the storage unit; extracting an orientationof the house by analyzing directions of the detected edges; andgenerating a polygon of an outline of the house by using linear lines ofthe extracted orientation of the house.
 8. The geospatial informationcreating method according to claim 7, further including the steps of:receiving specification of a position by an operator for the house to beextract the footprint; determining a house area of an extent of theextraction of the edges by periodically extract neighboring pixels moresimilar than a threshold before extraction of the edges; and changingthe threshold to decrease, and extracting again neighboring pixels moresimilar than the changed threshold in the case where the determinedhouse area does not contain the specified position.
 9. The geospatialinformation creating method according to claim 7, further including thesteps of: determining straight line segments to which the detected edgesfit; defining the orientation of the house based on angles of thedetermined straight line segments; classifying the determined straightline segments into those in the defined orientation of the house, andthose in a direction orthogonal thereto; generating grids at positionsat which the classified straight line segments exist; and generating thepolygon of the outline of the house according to the generated grids.10. The geospatial information creating method according to claim 9,further including the steps of: calculating a color spectrum in eachgrid cell segmented by the generated grids, and calculating a similarityindex of neighboring grid cells based on the calculated color spectrum;and merging grid cells having the calculated similarity index largerthan a predetermined threshold to generate a rectilinear polygon patchcorresponding to the footprint of the house surrounded by linear lines.11. The geospatial information creating method according to claim 9,further including the steps of: determining a first frequency of aposition of the determined straight line segments which is weighted by alength of the determined straight line segments, and generating a gridline constructing the grid at a position having the first frequencylarger than a predetermined threshold; and calculating the firstfrequency so that the first frequency of an straight line segments closeto the generated grid line is added to the first frequency of the gridline.
 12. The geospatial information creating method according to claim9, further including the steps of determining a second frequency of theangle of the determined straight line segments which is weighted by alength of the determined line, and defining an angle high in the secondfrequency as the orientation of the house.